Systems, methods and apparatus for prevention of injury

ABSTRACT

Systems, methods and apparatus are provided through which in some implementations, a system calculates a potential for ACL injury of a subject using skeletal tracking and calculating joint angles at different time points from the motion, angle of the knee joint, jumping/landing mechanics and/or balance of the subject.

FIELD

This disclosure relates generally to medical image analysis, injuryprevention, and more particularly to anterior cruciate ligament (ACL)ligament analysis.

BACKGROUND

The anterior cruciate ligament (ACL) ligament is one of a pair ofcruciate ligaments in the human knee. The two ligaments are also calledcruciform ligaments, as they are arranged in a crossed formation. In thequadruped stifle joint, based on its anatomical position, the ACLligament is also referred to as the cranial cruciate ligament.

ACL injuries (such as tears) are frequent injuries, making ACLreconstruction one of the most commonly performed orthopedic surgeries.Despite significant advances in surgical techniques over the years, ACLreconstruction remains fraught with frequent complications like graftrupture and instability that lead to clinical failure, and ultimatelyosteoarthritis. Risk factors that have been associated with ACL injuryinclude the size of the femoral notch, the diameter of the ACL, themechanical axis of the lower extremities and other specific anatomicalfeatures. Although ACL tears can occur in many sports, ACL tears areexceedingly prevalent in pivoting sports such as soccer. Female athletesare particularly prone to ACL, in fact, female athletes tear their ACLat least two and up to eight times more frequently than male athletes.Multiple factors have been associated with the higher risk of ACL injuryin women athletes, such as the size of the femoral notch, the diameterof the ACL, the mechanical axis of the lower extremities, hormonalstatus and increased knee laxity. Many of these factors are notmodifiable, other factors such as core stability, quadriceps dominanceand jumping/landing mechanics in women are different than in men andthought to be important risk factors for ACL injuries. These differencesin neuromuscular coordination put the ACL under conditions of increasedstrain in female athletes. Neuromuscular training programs have beenshown to decrease injury rates in populations of female athletes.However, the adoption of these programs is far from uniform norwidespread. One of the current solutions to study and prevent ACLinjuries are expensive gait motion analysis laboratories, which placemany reflective markers on patients, record video and then performanalysis which can take upward of 2 hours per patient. These motionanalysis laboratories are extremely expensive. The use of these motionanalysis systems has shown that jumping dynamics can reliably predictrisk of knee injury, but such systems are not portable, and are for themost part are only available in academic institutions for well-equippedresearch purposes and not for population general use.

BRIEF DESCRIPTION

The above-mentioned shortcomings, disadvantages and problems areaddressed herein, which will be understood by reading and studying thefollowing specification.

Given the sheer amount of ACL injuries, the identifiable neuromuscularrisk factors and the vulnerability, in particular, of female athletes tothis devastating injury with long term consequences, I have developedinjury prevention technique and system software that are powerful andeasy to use and adopt using commercially available motion capturehardware that is readily available and soon to be found in mostsmartphones. The system software can serve as a screening tool fortherapists, coaches, medical professionals, even parents to identifyathletes with specific neuromuscular imbalances predisposing them to ACLinjury. The system software recognizes and flags athletes at high riskof ACL tears. The application alerts the athletes and those involved inhis/her success to the need for participation in a corrective exerciseprogram/neuromuscular training program. The system software alsoassesses a successful correction of the risky neuromuscularcoordination. The system software also guides return to play decisionsparticularly in the context of recovering from ACL reconstructionsurgery.

I have pioneered a portable, low-cost and user-friendly solution thatcan provide a useful screening tool for athletes at risk of injury andthat can be adopted by medical professionals but also by coaches,parents and athletes themselves, for widespread ACL injury prevention atthe global level.

In one aspect, a system calculates an injury risk score for a subjectusing skeletal tracking or pose recognition and calculating joint anglesat different time points from the motion, angle of the knee joint,jumping/landing mechanics, balance of the subject and if the score isover a pre-determined threshold, then flags the subject as being “atrisk”. Additional variables such as biological gender of the subject(male vs female), sport(s) played by the subject, and/or previousinjury(ies) of the subject; can also be added.

In another aspect, a computer-based system determines a potential injuryto a knee anterior cruciate ligament of a subject, the computer-basedsystem includes a motion sensing input device and a computer having aprocessor, a memory and input/output capability, the memory beingconfigured to perform skeletal tracking and calculating joint angles atdifferent time points and being configured to analyze the motion of thesubject, angle of the knee joint of the subject, jumping and landingmechanics of the subject and balance of the subject in reference to theskeletal tracking and calculating joint angles at different time points,and to determine the potential injury to the knee anterior cruciateligament of the subject.

Apparatus, systems, and methods of varying scope are described herein.In addition to the aspects and advantages described in this summary,further aspects and advantages will become apparent by reference to thedrawings and by reading the detailed description that follows.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an anterior cruciate ligament injuryscreening system, according to an implementation.

FIG. 2 is a block diagram of an anterior cruciate ligament injuryscreening system, according to an implementation that includes aMicrosoft Kinect for Xbox One.

FIG. 3 is a block diagram of a hand-held imaging system, according to ansmartphone implementation.

FIG. 4 is a flowchart of a method to determine potential injury to a ACLof a subject (expressed as a ACL risk score) from motion sensory imagesof a knee of the subject, according to an implementation.

FIG. 5 is a block diagram of a solid-state image transducer, accordingto an implementation.

FIG. 6 is a block diagram of a hardware and operating environment inwhich different implementations can be practiced.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawings that form a part hereof, and in which is shown byway of illustration specific implementations which may be practiced.These implementations are described in sufficient detail to enable thoseskilled in the art to practice the imamplementations, and it is to beunderstood that other implementations may be utilized and that logical,mechanical, electrical and other changes may be made without departingfrom the scope of the implementations. The following detaileddescription is, therefore, not to be taken in a limiting sense.

The detailed description is divided into five sections. In the firstsection, a system level overview is described. In the second section,apparatus of implementations are described. In the third section,implementations of methods are described. In the fourth section, ahardware and the operating environment in conjunction with whichimplementations may be practiced are described. Finally, in the fifthsection, a conclusion of the detailed description is provided.

The system 100 in FIG. 1 , the system 200 in FIG. 2 , the hand-heldimaging system 300 in FIG. 3 , the method 400 in FIG. 4 and the hardwareand operating environment 600 in FIG. 6 do not include a motion profilethat further includes ranges of node angle value, do not comprise rangesof node angle value in a motion profile, do not comprise a motionprofile that has ranges of node angle value or that has displacementexperienced for one or more exercises, do not comprise a database ofpreviously recorded motion profiles, and do not determine arehab/treatment schedule.

System Level Overview

FIG. 1 is a block diagram of an anterior cruciate ligament injuryscreening system 100, according to an implementation. System 100provides a determination of potential of injury to a knee anteriorcruciate ligament of a subject.

System 100 includes a motion sensing input device 102 that detectsmotion sensory electromagnetic energy 104 of a subject 106 and thatgenerates a motion sensory image 110 of the subject 106. In oneimplementation, the motion sensory image 110 is captured while thesubject performs a standardized drop-vertical-jump procedure, astandardized tuck jump procedure or a standardized triple single leg hopprocedure, which are commonly utilized in the assessment of knee injury.The motion sensing input device 102 is operably coupled to a computer112. In some implementations, the motion sensing input device 102records 4-5 seconds of motion sensory imagery, at 30 hz (roughly 15frames per second), which is 60-75 frames of useful information.

In some implementations, the motion sensing input device 102 includes anRGB camera, a depth sensor and a microphone array running proprietarysoftware which provide full-body 3D motion capture, facial recognitionand voice recognition capabilities. Some implementations of themicrophone array of the motion sensing input device 102 enable acousticsource localization and ambient noise suppression. Some implementationsof the depth sensor of the motion sensing input device 102 include aninfrared laser projector combined with a monochrome CMOS sensor, whichcaptures video data in 3D under any ambient light conditions, in whichthe sensing range of the depth sensor is adjustable, and the depthsensor can be calibrated based on the duration of the motion sensorycapture and the surrounding physical environment, accommodating for thepresence of furniture or other obstacles. Some implementations themotion sensing input device 102 provides the motion sensing image 110 ofmore than one subject 106 in the field of view of the motion sensinginput device 102. Some implementations the motion sensing input device102 outputs video at a frame rate of ≈9 Hz to 30 Hz depending onresolution, in which the default RGB video stream uses 8-bit VGAresolution (640×480 pixels) with a Bayer color filter, using componentscapable of resolutions up to 1280×1024 (at a lower frame rate) and othercolour formats such as UYVY, wherein the monochrome depth sensing videostream is in VGA resolution (640×480 pixels) with 11-bit depth, whichprovides 2,048 levels of sensitivity, whereby the view is streamed fromthe IR camera directly (i.e.: before it has been converted into a depthmap) as 640×480 video, or 1280×1024 at a lower frame rate; providing apractical ranging limit of 1.2-3.5 m (3.9-11.5 ft) distance. The arearequired to perform motion by the subject 106 is roughly 6 m2, althoughthe sensor can maintain tracking through an extended range ofapproximately 0.7-6 m (2.3-19.7 ft). The angular field of view can be57° horizontally and 43° vertically, while implementations having amotorized pivot is capable of tilting the sensor up to 27° either up ordown. In some implementations, a horizontal field of the motion sensinginput device 102 at the minimum viewing distance of ≈0.8 m (2.6 ft) istherefore ≈87 cm (34 in), and the vertical field is ≈63 cm (25 in),resulting in a resolution of just over 1.3 mm (0.051 in) per pixel. Insome implementations, the microphone array features four microphonecapsules and operates with each channel processing 16-bit audio at asampling rate of 16 kHz.

The computer 112 includes a processor 114, memory 116 and input/outputcircuits 118. The memory 116 is configured with instructions 120 thatthe processor 114 can perform for the computer 112 to interact with themotion sensing input device 102 as well as a custom user interface. Forexample, a numerical analysis module of the instructions 120 identify anumber of appropriate frames of recording to perform vector calculus forthe coronal and sagittal angles measured during the standardizeddrop-vertical-jump procedure. The memory is 116 configured withinstructions 120 that the processor 114 can perform to calculate motionand the memory 116 is configured with instructions 120 that theprocessor 114 can perform to analyze motion of the subject 106, angle ofthe knee joint of the subject, jumping and landing mechanics of thesubject 106, balance of the subject 106 in reference to the skeletaltracking and calculating joint angles at different time points, thatgenerates a determination of potential injury 122 to a knee anteriorcruciate ligament of a subject 118. Additional variables such asbiological gender of the subject 106 (male vs female), sport(s) playedby the subject 106, and/or previous injury(ies) of the subject 106 canbe input to improve determination of the potential injury to the kneeanterior cruciate ligament of the subject 106. The subject 106 has nomarkers attached to the subject 106.

While the system 100 is not limited to any particular motion sensinginput device 102, motion sensory electromagnetic energy 104, subject106, motion sensory image 110, computer 112, processor 114, memory 116,input/output circuits 118, instructions 120 or determination ofpotential injury 122 to a knee anterior cruciate ligament of thesubject; for sake of clarity, a simplified motion sensing input device102, motion sensory electromagnetic energy 104, subject 106, motionsensory image 110, computer 112, processor 114, memory 116, input/outputcircuits 118, instructions 120 and determination of potential injury 122to a knee anterior cruciate ligament of the subject are described.

Apparatus Implementations

FIG. 2 is a block diagram of an anterior cruciate ligament injuryscreening system 200, according to an implementation that includes aMicrosoft Kinect for Xbox One. System 200 provides a determination ofpotential injury to a knee anterior cruciate ligament of a subject.

System 200 includes a 3D video recording input device 202 such as theMicrosoft Kinect for Xbox One that detects motion sensoryelectromagnetic energy 104 of a subject 106 and that generates a motionsensory image 204 of the subject 106. In one implementation, the motionsensory image 204 is captured while the subject performs a standardizeddrop-vertical-jump procedure which is commonly utilized in theassessment of knee injury. The 3D video recording input device 202 isoperably coupled to a computer 206. In one implementation, the 3D videorecording input device 202 records 4-5 seconds of imagery, at 30 hz(roughly 15 frames per second), which is 60-75 frames of usefulinformation. In other implementations, a Wii Remote™, Wii Remote Plus™,Wii Balance Board™ for the Wii™ and Wii U™, PlayStation Move™,PlayStation Eye™ for the PlayStation 3™ or PlayStation Camera™ for thePlayStation 4™ is used in place of the 3D video recording input device202.

The computer 206 includes a processor 208, memory 210 and input/outputcircuits 212. The memory 210 is configured with instructions 214 thatthe processor 208 can perform to interact with a sensor of the MicrosoftKinect for Xbox One; as well as a custom user interface. The memory 210is configured with instructions 214 that the processor 208 can performin a numerical analysis module of the instructions 214 to identify anumber of appropriate frames of recording to perform vector calculus forcoronal and sagittal angles measured during the standardizeddrop-vertical-jump procedure; and instructions 214 to analyze motion ofthe subject 106, angle of the knee joint of the subject, jumping andlanding mechanics of the subject 106, balance of the subject 106, inreference to the skeletal tracking and calculating joint angles atdifferent time points, and to determine the potential injury to the kneeanterior cruciate ligament of the subject 106, that generates adetermination 216 of potential injury to a knee anterior cruciateligament of the subject 106. Additional variables such as biologicalgender of the subject 106 (male vs female), sport(s) played by thesubject 106, and/or previous injury(ies) of the subject 106; can also beinput.

While the system 200 is not limited to any particular Microsoft Kinectfor Xbox One, motion sensory electromagnetic energy 104, subject 106,motion sensory image 204, computer 206, a processor 208, memory 210,input/output circuits 212, instructions 214 and determination 216; forsake of clarity, a simplified Microsoft Kinect for Xbox One, motionsensory electromagnetic energy 104, subject 106, motion sensory image204, computer 206, a processor 208, memory 210, input/output circuits212, instructions 214 and determination 216 are described.

In the previous section, a system level overview of the operation of animplementation was described. In this section, the particular apparatusof such an implementation are described by reference to a series ofdiagrams.

FIG. 3 is a block diagram of a hand-held imaging system 300, accordingto a smartphone implementation. The hand-held imaging system 300includes a number of modules such as a main processor 302 that controlsthe overall operation of the hand-held imaging system 300. Communicationfunctions, including data and voice communications, can be performedthrough a communication subsystem 304. The communication subsystem 304receives messages from and sends messages to wireless networks 305. Inother implementations of the hand-held imaging system 300, thecommunication subsystem 304 can be configured in accordance with theGlobal System for Mobile Communication (GSM), General Packet RadioServices (GPRS), Enhanced Data GSM Environment (EDGE), Universal MobileTelecommunications Service (UMTS), data-centric wireless networks,voice-centric wireless networks, and dual-mode networks that can supportboth voice and data communications over the same physical base stations.Combined dual-mode networks include, but are not limited to, CodeDivision Multiple Access (CDMA) or CDMA2000 networks, GSM/GPRS networks(as mentioned above), third-generation (3G) networks like EDGE and UMTS,4G and 5G. Some other examples of data-centric networks include Mobitex™and DataTAC™ network communication systems. Examples of othervoice-centric data networks include Personal Communication Systems (PCS)networks like GSM and Time Division Multiple Access (TDMA) systems.

The wireless link connecting the communication subsystem 304 with thewireless network 305 represents one or more different Radio Frequency(RF) channels. With newer network protocols, these channels are capableof supporting both circuit switched voice communications and packetswitched data communications.

The main processor 302 also interacts with additional subsystems such asa Random Access Memory (RAM) 306, a flash memory 308, a display 310, anauxiliary input/output (I/O) subsystem 312, a data port 314, a keyboard316, a speaker 318, a microphone 320, short-range communicationssubsystem 322 and other device subsystems 324. In some implementations,the flash memory 308 includes a hybrid femtocell/Wi-Fi® protocol stack309. The hybrid femtocell/Wi-Fi® protocol stack 309 supportsauthentication and authorization between the hand-held imaging system300 into a shared Wi-Fi® network and both a 3G, 4G or 5G mobilenetworks.

The hand-held imaging system 300 can transmit and receive communicationsignals over the wireless network 305 after required networkregistration or activation procedures have been completed. Networkaccess is associated with a subscriber or user of the hand-held imagingsystem 300. User identification information can also be programmed intothe flash memory 308.

The hand-held imaging system 300 is a battery-powered device andincludes a battery interface 332 for receiving one or more batteries330. In one or more implementations, the battery 330 can be a smartbattery with an embedded microprocessor. The battery interface 332 iscoupled to a regulator 333, which assists the battery 330 in providingpower V+ to the hand-held imaging system 300. Future technologies suchas micro fuel cells may provide the power to the hand-held imagingsystem 300.

The hand-held imaging system 300 also includes an operating system 334and modules 336 to 350 which are described in more detail below. Theoperating system 334 and the modules 336 to 350 that are executed by themain processor 302 are typically stored in a persistent nonvolatilemedium such as the flash memory 308, which may alternatively be aread-only memory (ROM) or similar storage element (not shown). Thoseskilled in the art will appreciate that portions of the operating system334 and the modules 336 to 350, such as specific device applications, orparts thereof, may be temporarily loaded into a volatile store such asthe RAM 306. Other modules can also be included.

The subset of modules 336 that control basic device operations,including data and voice communication applications, will normally beinstalled on the hand-held imaging system 300 during its manufacture.Other modules include a message application 338 that can be any suitablemodule that allows a user of the hand-held imaging system 300 totransmit and receive electronic messages. Various alternatives exist forthe message application 338 as is well known to those skilled in theart. Messages that have been sent or received by the user are typicallystored in the flash memory 308 of the hand-held imaging system 300 orsome other suitable storage element in the hand-held imaging system 300.In one or more implementations, some of the sent and received messagesmay be stored remotely from the hand-held imaging system 300 such as ina data store of an associated host system with which the hand-heldimaging system 300 communicates.

The modules can further include a device state module 340, a PersonalInformation Manager (PIM) 342, and other suitable modules (not shown).The device state module 340 provides persistence, i.e. the device statemodule 340 ensures that important device data is stored in persistentmemory, such as the flash memory 308, so that the data is not lost whenthe hand-held imaging system 300 is turned off or loses power.

The PIM 342 includes functionality for organizing and managing dataitems of interest to the user, such as, but not limited to, e-mail,contacts, calendar events, voice mails, appointments, and task items. APIM application has the ability to transmit and receive data items viathe wireless network 305. PIM data items may be seamlessly integrated,synchronized, and updated via the wireless network 305 with thehand-held imaging system 300 subscriber's corresponding data itemsstored and/or associated with a host computer system. This functionalitycreates a mirrored host computer on the hand-held imaging system 300with respect to such items.

The hand-held imaging system 300 also includes a connect module 344, andan IT policy module 346. The connect module 344 implements thecommunication protocols that are required for the hand-held imagingsystem 300 to communicate with the wireless infrastructure and any hostsystem, such as an enterprise system, with which the hand-held imagingsystem 300 is authorized to interface.

The connect module 344 includes a set of APIs that can be integratedwith the hand-held imaging system 300 to allow the hand-held imagingsystem 300 to use any number of services associated with the enterprisesystem. The connect module 344 allows the hand-held imaging system 300to establish an end-to-end secure, authenticated communication pipe withthe host system. A subset of applications for which access is providedby the connect module 344 can be used to pass IT policy commands fromthe host system to the hand-held imaging system 300. This can be done ina wireless or wired manner. These instructions can then be passed to theIT policy module 346 to modify the configuration of the hand-heldimaging system 300. Alternatively, in some cases, the IT policy updatecan also be done over a wired connection.

The IT policy module 346 receives IT policy data that encodes the ITpolicy. The IT policy module 346 then ensures that the IT policy data isauthenticated by the hand-held imaging system 300. The IT policy datacan then be stored in the RAM 306 in its native form. After the ITpolicy data is stored, a global notification can be sent by the ITpolicy module 346 to all of the applications residing on the hand-heldimaging system 300. Applications for which the IT policy may beapplicable then respond by reading the IT policy data to look for ITpolicy rules that are applicable.

The programs 337 can also include an ACL risk score generator 350. Asolid-state image transducer 354 captures images 356 and the ACL riskscore generator 350 generates the ACL risk score(s) 352. In someimplementations, the ACL risk score(s) 352 are expressed as “high risk”“low risk”, or “red light” “green light”. In one implementation, thesolid-state image transducer 354 records 4-5 seconds of imagery, at 120hz (roughly 60 frames per second), which is 60-75 frames of usefulinformation. In some implementations, the hand-held imaging system 300includes the solid-state image transducer 354 in an internal or externalcamera module that performs the functions of the motion sensing inputdevice 102.

The ACL risk score generator 350 performs substantially similarfunctions in FIG. 1 as the instructions 120 that the processor 114 canperform to analyze motion of the subject 106, angle of the knee joint ofthe subject, jumping and landing mechanics of the subject 106, balanceof the subject 106 in reference to the skeletal tracking and calculatingjoint angles at different time points, and to determine the potentialinjury to the knee anterior cruciate ligament of the subject 106, thatgenerates the determination of potential injury 122 to a knee anteriorcruciate ligament of a subject 118. Additional variables such asbiological gender of the subject 106 (male vs female), sport(s) playedby the subject 106, and/or previous injury(ies) of the subject 106 canalso be input. The ACL risk score generator 350 performs substantiallysimilar functions in FIG. 2 as the instructions 214 that the processor208 can perform to analyze motion of the subject 106, angle of the kneejoint of the subject, jumping and landing mechanics of the subject 106,balance of the subject 106 in reference to the skeletal tracking andcalculating joint angles at different time points, and to determine thepotential injury to the knee anterior cruciate ligament of the subject106, that generates a determination 216 of potential injury to a kneeanterior cruciate ligament of the subject 106. Additional variables suchas biological gender of the subject 106 (male vs female), sport(s)played by the subject 106, and/or previous injury(ies) of the subject106 can also be input.

In some implementations, the ACL risk score generator 350 performs thesame functions as instructions that the main processor 302 can performto analyze motion of the subject 106, angle of the knee joint of thesubject, jumping and landing mechanics of the subject 106, balance ofthe subject 106 in reference to the skeletal tracking and calculatingjoint angles at different time points—to determine the potential injuryto the knee anterior cruciate ligament of the subject 106 from theimages 356 received from motion sensory solid state image transducer354. Additional variables such as biological gender of the subject 106(male vs female), sport(s) played by the subject 106, and/or previousinjury(ies) of the subject 106 can also be input. In someimplementations, the hand-held imaging system 300 includes no ACL riskscore generator 350 and the determined ACL risk scores are receivedthrough the data port 314, the communication subsystem 304 or theshort-range communications subsystem 322 from another electronic devicesuch as the computer 112 in FIG. 1 or computer 206 in FIG. 2 .

In some implementations, the ACL risk score generator 350 performsmachine learning in the determination of the potential injury to theknee anterior cruciate ligament of the subject 106 as more and more setsof images 356 are processed. Machine learning in FIG. 3 uses algorithmsand statistical models to perform the functions of the ACL risk scoregenerator 350, relying on patterns and inference instead. The machinelearning algorithms build a mathematical model based on sample data,known as “training data”. Machine learning tasks are classified intoseveral broad categories. In supervised machine learning, the algorithmbuilds a mathematical model from a set of data that contains both theinputs and the desired outputs. For example, if the task weredetermining whether an image contained a certain object, the trainingdata for a supervised machine learning algorithm would include imageswith and without that object (the input), and each image would have alabel (the output) designating whether it contained the object. Inspecial cases, the input may be only partially available, or restrictedto special feedback. Semi-supervised machine learning algorithms developmathematical models from incomplete training data, where a portion ofthe sample input doesn't have labels. Classification algorithms andregression algorithms are types of supervised machine learning.Classification algorithms are used when the outputs are restricted to alimited set of values. For a classification algorithm that filtersemails, the input would be an incoming email, and the output would bethe name of the folder in which to file the email. For an algorithm thatidentifies spam emails, the output would be the prediction of either“spam” or “not spam”, represented by the Boolean values true and false.Regression algorithms are named for their continuous outputs, meaningthey may have any value within a range. Examples of a continuous valueare the temperature, length, or price of an object. In unsupervisedmachine learning, the algorithm builds a mathematical model from a setof data which contains only inputs and no desired output labels.Unsupervised machine learning algorithms are used to find structure inthe data, like grouping or clustering of data points. Unsupervisedmachine learning can discover patterns in the data, and can group theinputs into categories, as in feature machine learning. Dimensionalityreduction is the process of reducing the number of “features”, orinputs, in a set of data. Active machine learning algorithms access thedesired outputs (training labels) for a limited set of inputs based on abudget, and optimize the choice of inputs for which it will acquiretraining labels. When used interactively, these can be presented to ahuman user for labeling. Reinforcement machine learning algorithms aregiven feedback in the form of positive or negative reinforcement in adynamic environment, and are used in autonomous vehicles or in machinelearning to play a game against a human opponent. Other specializedalgorithms in machine learning include topic modeling, where the ACLrisk score generator 350 is given a set of natural language documentsand finds other documents that cover similar topics. Machine learningalgorithms can be used to find the unobservable probability densityfunction in density estimation problems. Meta machine learningalgorithms learn their own inductive bias based on previous experience.In developmental robotics, robot machine learning algorithms generatetheir own sequences of machine learning experiences, also known as acurriculum, to cumulatively acquire new skills through self-guidedexploration and social interaction with humans. These robots useguidance mechanisms such as active machine learning, maturation, motorsynergies, and imitation.

An ACL risk score 352 is generated by the ACL risk score generator 354,or is received from an external source, is then is displayed by display310 or transmitted by the communication subsystem 304 or the short-rangecommunications subsystem 322, enunciated by the speaker 318 or stored bythe flash memory 308.

Other types of modules can also be installed on the hand-held imagingsystem 300. These modules can be third party modules, which are addedafter the manufacture of the hand-held imaging system 300. Examples ofthird party applications include games, calculators, utilities, andadditional imaging devices, etc.

The additional applications can be loaded onto the hand-held imagingsystem 300 through of the wireless network 305, the auxiliary I/Osubsystem 312, the data port 314, the short-range communicationssubsystem 322, or any other suitable device subsystem 324. Thisflexibility in application installation increases the functionality ofthe hand-held imaging system 300 and may provide enhanced on-devicefunctions, communication-related functions, or both. For example, securecommunication applications enable electronic commerce functions andother such financial transactions to be performed using the hand-heldimaging system 300.

The data port 314 enables a subscriber to set preferences through anexternal device or module and extends the capabilities of the hand-heldimaging system 300 by providing for information or module downloads tothe hand-held imaging system 300 other than through a wirelesscommunication network. The alternate download path may, for example, beused to load an encryption key onto the hand-held imaging system 300through a direct and thus reliable and trusted connection to providesecure device communication.

The data port 314 can be any suitable port that enables datacommunication between the hand-held imaging system 300 and anothercomputing device. The data port 314 can be a serial or a parallel port.In some instances, the data port 314 can be a USB port that includesdata lines for data transfer and a supply line that can provide acharging current to charge the battery 330 of the hand-held imagingsystem 300.

The short-range communications subsystem 322 provides for communicationbetween the hand-held imaging system 300 and different systems ordevices, without the use of the wireless network 305. For example, theshort-range communications subsystem 322 may include a motion sensorydevice and associated circuits and modules for short-rangecommunication. Examples of short-range communication standards includestandards developed by the Infrared Data Association (IrDA), Bluetooth®,and the 802.11 family of standards developed by IEEE. In otherimplementations, Zigbee® or Z-Wave® can be used instead of Bluetooth®.

Bluetooth® is a wireless technology standard for exchanging data overshort distances (using short-wavelength radio transmissions in the ISMband from 2400-2480 MHz) from fixed and mobile devices, creatingpersonal area networks (PANs) with high levels of security. Created bytelecom vendor Ericsson in 1994, Bluetooth® was originally conceived asa wireless alternative to RS-232 data cables. Bluetooth® can connectseveral devices, overcoming problems of synchronization. Bluetooth®operates in the range of 2400-2483.5 MHz (including guard bands), whichis in the globally unlicensed Industrial, Scientific and Medical (ISM)2.4 GHz short-range radio frequency band. Bluetooth® uses a radiotechnology called frequency-hopping spread spectrum. The transmitteddata is divided into packets and each packet is transmitted on one ofthe 79 designated Bluetooth® channels. Each channel has a bandwidth of 1MHz. The first channel starts at 2402 MHz and continues up to 2480 MHzin 1 MHz steps. The first channel usually performs 1600 hops per second,with Adaptive Frequency-Hopping (AFH) enabled. Originally Gaussianfrequency-shift keying (GFSK) modulation was the only modulation schemeavailable; subsequently, since the introduction of Bluetooth® 2.0+EDR,π/4-DQPSK and 8DPSK modulation may also be used between compatibledevices. Devices functioning with GFSK are said to be operating in basicrate (BR) mode where an instantaneous data rate of 1 Mbit/s is possible.The Bluetooth® Core Specification provides for the connection of two ormore piconets to form a scatternet, in which certain devicessimultaneously play the master role in one piconet and the slave role inanother. At any given time, data can be transferred between the masterand one other device (except for the little-used broadcast mode. Themaster chooses which slave device to address; typically, the masterswitches rapidly from one device to another in a round-robin fashion.Since the master chooses which slave to address, whereas a slave is (intheory) supposed to listen in each receive slot, being a master is alighter burden than being a slave. Being a master of seven slaves ispossible; being a slave of more than one master is difficult. Many ofthe services offered over Bluetooth® can expose private data or allowthe connecting party to control the Bluetooth® device. For securityreasons it is necessary to be able to recognize specific devices andthus enable control over which devices are allowed to connect to a givenBluetooth® device. At the same time, it is useful for Bluetooth® devicesto be able to establish a connection without user intervention (forexample, as soon as the Bluetooth® devices of each other are in range).To resolve this conflict, Bluetooth® uses a process called bonding, anda bond is created through a process called pairing. The pairing processis triggered either by a specific request from a user to create a bond(for example, the user explicitly requests to “Add a Bluetooth®device”), or the pairing process is triggered automatically whenconnecting to a service where (for the first time) the identity of adevice is required for security purposes. These two cases are referredto as dedicated bonding and general bonding respectively. Pairing ofteninvolves some level of user interaction; this user interaction is thebasis for confirming the identity of the devices.

In use, a received signal such as a text message, an e-mail message, orweb page download will be processed by the communication subsystem 304and input to the main processor 302. The main processor 302 will thenprocess the received signal for output to the display 310 oralternatively to the auxiliary I/O subsystem 312. A subscriber may alsocompose data items, such as e-mail messages, for example, using thekeyboard 316 in conjunction with the display 310 and possibly theauxiliary I/O subsystem 312. The auxiliary I/O subsystem 312 may includedevices such as: a touch screen, mouse, track ball, infrared fingerprintdetector, or a roller wheel with dynamic button pressing capability. Thekeyboard 316 is preferably an alphanumeric keyboard and/ortelephone-type keypad. However, other types of keyboards may also beused. A composed item may be transmitted over the wireless network 305through the communication subsystem 304.

For voice communications, the overall operation of the hand-held imagingsystem 300 is substantially similar, except that the received signalsare output to the speaker 318, and signals for transmission aregenerated by the microphone 320. Alternative voice or audio I/Osubsystems, such as a voice message recording subsystem, can also beimplemented on the hand-held imaging system 300. Although voice or audiosignal output is accomplished primarily through the speaker 318, thedisplay 310 can also be used to provide additional information such asthe identity of a calling party, duration of a voice call, or othervoice call related information.

Method Implementations

In the previous section, apparatus of the operation of an implementationwas described. In this section, the particular methods performed byprocessor 114 in FIG. 1 , processor 208 in FIG. 2 , main processor 302in FIG. 3 and processing unit 604 in FIG. 6 of such an implementationare described by reference to a flowchart.

FIG. 4 is a flowchart of a method 400 to determine potential of injuryto the ACL of a subject (expressed as a ACL risk score) from motionsensory images of a knee of the subject, according to an implementation.

Method 400 is one implementation of the process performed by theinstructions 120 in FIG. 1 , instructions 214 in FIG. 2 and the ACL riskscore generator 350 in FIG. 3 .

Method 400 includes skeletal tracking and calculating joint angles atdifferent time points 402 between the ACL images and known angles of theknee that are associated with increased risk of ACL injury. Examples ofthe ACL images are motion sensory image 110 in FIG. 1 , motion sensoryimage 204 in FIG. 2 and images 356 in FIG. 3 .

Method 400 also includes analyzing 404 a motion of the subject in theimages, analyzing the angle of the joints, trunk and lower extremitiesof the subject, analyzing jumping and landing mechanics of the subject,analyzing balance of the subject the results of the skeletal trackingand calculating joint angles at different time points.

Method 400 also includes determining 406 a potential injury to the ACLof the subject 106, which in some implementations is performed as(measured angle−uninjured angle)/(injured angle−uninjured angle).

In some implementations, method 400 is implemented as a sequence ofcomputer instructions which, when executed by a processor (such asprocessor 114 in FIG. 1 , processor 208 in FIG. 2 , main processor 302in FIG. 3 and processor 604 in FIG. 6 ), cause the processor to performthe respective method. In other implementations, method 400 isimplemented as a computer-accessible medium having executableinstructions capable of directing a processor, such as processor 604 inFIG. 6 , to perform the respective method. In varying implementations,the medium is a magnetic medium, an electronic medium, or an opticalmedium.

While method 400 is not limited to any particular knee ACL, subject,imaging apparatus or processor; for sake of clarity; a simplified kneeACL, subject, imaging apparatus and processor are described.

Hardware and Operating Environment

FIG. 5 is a block diagram of a solid-state image transducer 500,according to an implementation. The solid-state image transducer 500 isone component of the motion sensing input device 102. The solid-stateimage transducer 500 is one component of the 3D video recording inputdevice 202 in a Microsoft Kinect for Xbox One. The solid-state imagetransducer 500 is one example of the solid-state image transducer 354.The solid-state image transducer 500 includes a great number ofphotoelectric elements, a.sub.1 . . . sub.1, a.sub.2 . . . sub.1, . . ., a.sub.mn, in the minute segment form, transfer gates TG1, TG2, . . . ,TGn responsive to a control pulse V.sub.φP for transferring the chargesstored on the individual photoelectric elements as an image signal tovertical shift registers VS1, VS2, . . . , VSn, and a horizontal shiftregister HS for transferring the image signal from the vertical shiftregisters VSs, through a buffer amplifier to an outlet. After theone-frame image signal is stored, the image signal is transferred tovertical shift register by the pulse V.sub.φP and the contents of thevertical shift registers VSs are transferred upward line by line inresponse to a series of control pulses V.sub.φV1, V.sub.φV2. During thetime interval between the successive two vertical transfer controlpulses, the horizontal shift register HS responsive to a series ofcontrol pulses V.sub.φH1, V.sub.φH2 transfers the contents of thehorizontal shift registers HSs in each line row by row to the right asviewed in FIG. 5. As a result, the one-frame image signal is formed byreading out the outputs of the individual photoelectric elements in suchorder.

FIG. 6 is a block diagram of a hardware and operating environment 600 inwhich different implementations can be practiced. The description ofFIG. 6 provides an overview of computer hardware and a suitablecomputing environment in conjunction with which some implementations canbe implemented. Implementations are described in terms of a computerexecuting computer-executable instructions. However, someimplementations can be implemented entirely in computer hardware inwhich the computer-executable instructions are implemented in read-onlymemory. Some implementations can also be implemented in client/servercomputing environments where remote devices that perform tasks arelinked through a communications network. Program modules can be locatedin both local and remote memory storage devices in a distributedcomputing environment.

Computer 602 includes a processor 604, commercially available fromIntel, Motorola, Cyrix and others. The computer 602 is oneimplementation of computer 112 in FIG. 1 and computer 206 in FIG. 2 .The processor 604 is one example of processor 114 in FIG. 1 andprocessor 208 in FIG. 2 . The computer 602 also includes system memory606 that includes random-access memory (RAM) 608 and read-only memory(ROM) 610. The RAM 608 and the ROM 610 are examples of the memory 116 inFIG. 1 and memory 210 in FIG. 2 . The computer 602 also includes one ormore mass storage devices 612; and a system bus 614 that operativelycouples various system components to the processing unit 604. The memory608 and 610, and mass storage devices 612, are types ofcomputer-accessible media. Mass storage devices 612 are morespecifically types of nonvolatile computer-accessible media and caninclude one or more hard disk drives, floppy disk drives, optical diskdrives, and tape cartridge drives. The processor 604 executes computerprograms stored on the computer-accessible media.

Computer 602 can be communicatively connected to the Internet 616 via acommunication device, such as modem 618. Internet 616 connectivity iswell known within the art. In one implementation, the modem 618 respondsto communication drivers to connect to the Internet 616 via what isknown in the art as a “dial-up connection.” In another implementation,the communication device is an Ethernet® or network adapter 620connected to a local-area network (LAN) 622 that itself is connected tothe Internet 616 via what is known in the art as a “direct connection”(e.g., T1 line, etc.).

A user enters commands and information into the computer 602 throughinput devices such as a keyboard (not shown) or a pointing device (notshown). The keyboard permits entry of textual information into computer602, as known within the art, and implementations are not limited to anyparticular type of keyboard. Pointing device permits the control of thescreen pointer provided by a graphical user interface (GUI) of operatingsystems such as versions of Microsoft Windows®. Implementations are notlimited to any particular pointing device. Such pointing devices includemice, touch pads, trackballs, remote controls and point sticks. Otherinput devices (not shown) can include a microphone, joystick, game pad,satellite dish, scanner, or the like.

In some implementations, computer 602 is operatively coupled to adisplay device 624. Display device 624 is connected to the system bus614 through a video adapter 626. Display device 624 permits the displayof information, including computer, video and other information, forviewing by a user of the computer. Implementations are not limited toany particular display device 624. Such display devices include cathoderay tube (CRT) displays (monitors), as well as flat panel displays suchas liquid crystal displays (LCD's). In addition to a monitor, computerstypically include other peripheral input/output devices such as printers(not shown). Speakers (not shown) provide audio output of signals.Speakers are also connected to the system bus 614.

Computer 602 can be operated using at least one operating system toprovide a graphical user interface (GUI) including a user-controllablepointer. Computer 602 can have at least one web browser applicationprogram executing within at least one operating system, to permit usersof computer 602 to access intranet or Internet world-wide-web pages asaddressed by Universal Resource Locator (URL) addresses. Examples ofbrowser application programs include Netscape Navigator® and MicrosoftInternet Explorer®.

The computer 602 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer628. These logical connections are achieved by a communication devicecoupled to, or a part of, the computer 602. Implementations are notlimited to a particular type of communications device. The remotecomputer 628 can be another computer, a server, a router, a network PC,a client, a peer device or other common network node. The logicalconnections depicted in FIG. 6 include the local-area network (LAN) 622and a wide-area network (WAN). Such networking environments arecommonplace in offices, enterprise-wide computer networks, intranets andthe Internet.

When used in a LAN-networking environment, the computer 602 and remotecomputer 628 are connected to the local network 622 through networkinterfaces or adapters 620, which is one type of communications device618. When used in a conventional WAN-networking environment, thecomputer 602 and remote computer 628 communicate with a WAN throughmodems. The modems, which can be internal or external, is connected tothe system bus 614. In a networked environment, program modules depictedrelative to the computer 602, or portions thereof, can be stored in theremote computer 628.

Computer 602 also includes an operating system 630 that can be stored onthe RAM 608 and ROM 610, and mass storage device 612, and is andexecuted by the processor 604. Examples of operating systems includeMicrosoft Windows®, Apple MacOS®, Linux®, UNIX®, providing capabilityfor supporting application programs 632 using, for example, code moduleswritten in the C++® computer programming language. Examples are notlimited to any particular operating system, however, and theconstruction and use of such operating systems are well known within theart.

Instructions can be stored via the mass storage devices 612 or systemmemory 606, including one or more application programs 632, otherprogram modules 634 and program data 636.

Computer 602 also includes power supply. Each power supply can be abattery.

Some implementations include computer instructions to generate andoperate a patient input screen that can be implemented in instructions120 in FIG. 1 , instructions 214 in FIG. 2 , or the ACL risk scoregenerator 350 in FIG. 3 or the instructions stored via the mass storagedevices 612 or system memory 606 in FIG. 6 .

Some implementations include computer instructions to generate andoperate the input capture device selection screen 800 that can beimplemented in instructions 120 in FIG. 1 , instructions 214 in FIG. 2 ,or the ACL risk score generator 350 in FIG. 3 or the instructions storedvia the mass storage devices 612 or system memory 606 in FIG. 6 .

Some implementations include computer instructions to generate andoperate an analysis module jump type selection screen, a recordationinitiation screen, a playback window screen, and a jump data andprediction screen that can be implemented in instructions 120 in FIG. 1, instructions 214 in FIG. 2 , or the ACL risk score generator 350 inFIG. 3 or the instructions stored via the mass storage devices 612 orsystem memory 606 in FIG. 6 .

CONCLUSION

An ACL injury risk determination system is described herein. A technicaleffect of the ACL injury risk determination system is determination ofpotential injury to the ACL of a subject (in some implementationsexpressed as a ACL risk score) from motion sensory images of a knee ofthe subject. Although specific implementations are illustrated anddescribed herein, it will be appreciated by those of ordinary skill inthe art that any arrangement which is calculated to achieve the samepurpose may be substituted for the specific implementations shown. Thisapplication is intended to cover any adaptations or variations. Forexample, although described in general terms, one of ordinary skill inthe art will appreciate that implementations can be made using any otherimaging technology that provides the required function.

In particular, one of skill in the art will readily appreciate that thenames of the methods and apparatus are not intended to limitimplementations. Furthermore, additional methods and apparatus can beadded to the components, functions can be rearranged among thecomponents, and new components to correspond to future enhancements andphysical devices used in implementations can be introduced withoutdeparting from the scope of implementations. One of skill in the artwill readily recognize that implementations are applicable to futureimaging devices, different processors, and new gait analyses, angle ofknee joint analyses, jumping/landing mechanics analyses, balanceanalyses of the subject, biological gender of the subject (male vsfemale), sport(s) played by the subject, and/or previous injury(ies) ofthe subject.

The terminology used in this application meant to include computingenvironments and alternate technologies which provide the samefunctionality as described herein.

The invention claimed is:
 1. A computer-based system for determining aprediction of future injury to a knee anterior cruciate ligament of asubject, the computer-based system comprising: a motion sensing inputdevice generating a plurality of images of the subject, the subjecthaving no markers attached to the subject, the motion sensing inputdevice having photoelectric elements, transfer gates that are responsiveto a control pulse for transferring the charges stored on the individualphotoelectric elements as an image signal to vertical shift registersand a horizontal shift register for transferring an image signal fromthe vertical shift registers through a buffer amplifier to an outlet,the plurality of images being captured at least 30 frames per second,each of the plurality of images having a resolution of at least 640×480pixels and 10 bit depth, having at least 2,048 levels of sensitivity; acomputer having a processor, a memory and input/output capability, theprocessor being configured to perform: a pose recognition or a skeletaltracking and calculating of joint angles at different time points; theprocessor being further configured to analyze from the plurality ofimages a motion of the subject, an angle of the knee joint of thesubject, jumping and landing mechanics of the subject, and a balance ofthe subject in reference to either the pose recognition, or the skeletaltracking and calculating of the joint angles at the different timepoints the processor being then further configured to determine theprediction of future injury to the knee anterior cruciate ligament ofthe subject based on the analysis of the plurality of images, whereinthe prediction of future injury to the knee anterior cruciate ligamentof the subject further comprises an ACL risk score; the processor beingthen further configured to display the ACL risk score by a display, ortransmit the ACL risk score by a communication subsystem, a transmit theACL risk score by short-range communications subsystem, enunciate theACL risk score by a speaker, or store the ACL risk score by a flashmemory into the processor; the processor being then further configuredto generate and operate a prediction screen displaying the ACL riskscore, wherein the ACL risk score is expressed on the prediction screenas “high risk” or “low risk”; wherein the processor is configured todetermine the prediction of future injury to the knee anterior cruciateligament by performing machine learning processes utilizing a modeltrained from a set of data that contains both desired inputs and desiredoutputs, in which the set of data for training the model includes imageswith and without the desired inputs, and each image includes a label forthe desired output designating whether the image contained a desiredtraining object; wherein the machine learning processes includesupervised regression learning which produces continuous outputs,wherein the continuous outputs are a continuous range of values for theACL risk score.
 2. The computer-based system of claim 1, wherein thecomputer-based system does not comprise a motion profile, the motionprofile further not comprising ranges of node angle value.
 3. Thecomputer-based system of claim 1, wherein the computer-based system doesnot comprise ranges of node angle value in a motion profile.
 4. Thecomputer-based system of claim 1, wherein the computer-based system doesnot comprise a motion profile, the motion profile further comprisingranges of node angle value or displacement experienced for one or moreexercises.
 5. The computer-based system of claim 1, wherein thecomputer-based system does not comprise a database of previouslyrecorded motion profiles.
 6. The computer-based system of claim 1,wherein the computer-based system does not determine a rehab/treatmentschedule.
 7. The computer-based system of claim 1, wherein the processorbeing further configured to perform the pose recognition or the skeletaltracking and the calculating joint angles at the different time pointsand being configured to analyze a gait of the subject, angle of the kneejoint of the subject and jumping and the landing mechanics of thesubject and the balance of the subject in further reference to othervariables including biological gender of the subject, sport played bythe subject, and/or previous injury of the subject to determine theprediction of future injury to the knee anterior cruciate ligament ofthe subject.
 8. A computer-based system for determining a prediction offuture injury to a knee anterior cruciate ligament of a subject, thecomputer-based system comprising: a motion sensing input devicegenerating a plurality of images of the subject, the subject having nomarkers attached to the subject, the motion sensing input device havingphotoelectric elements, transfer gates that are responsive to a controlpulse for transferring the charges stored on the individualphotoelectric elements as an image signal to vertical shift registersand a horizontal shift register for transferring an image signal fromthe vertical shift registers through a buffer amplifier to an outlet,the plurality of images being captured at least 30 frames per second,each of the plurality of images having a resolution of at least 640×480pixels and 10 bit depth and having at least 2,048 levels of sensitivity;a computer having a processor, a memory and input/output capability, theprocessor being configured to perform: a pose recognition or a skeletaltracking and calculating of joint angles at different time points; theprocessor being further configured to analyze from the plurality ofimages a motion of the subject, an angle of the knee joint of thesubject, jumping and landing mechanics of the subject in reference toeither the pose recognition, or the skeletal tracking and calculating ofthe joint angles at the different time points based on the analysis ofthe plurality of images; the processor being then further configured todetermine the prediction of future injury to the knee anterior cruciateligament of the subject based on the analysis of the plurality ofimages, wherein the prediction of future injury to the knee anteriorcruciate ligament of the subject further comprises an ACL risk score;the processor being then further configured to save the ACL risk scoreinto the memory as “at risk”; and the processor being then furtherconfigured to generate and operate a prediction screen displaying theACL risk score, wherein the ACL risk score is expressed on theprediction screen as “high risk” or “low risk”; wherein the processor isconfigured to determine the prediction of future injury to the kneeanterior cruciate ligament by performing machine learning processesutilizing a model trained from a set of data that contains both desiredinputs and desired outputs, in which the set of data for training themodel includes images with and without the desired inputs, and eachimage includes a label for the desired output designating whether theimage contained a desired training object; wherein the machine learningprocesses include supervised regression learning which producescontinuous outputs, wherein the continuous outputs are a continuousrange of values for the ACL risk score.
 9. The computer-based system ofclaim 8, wherein the computer-based system does not comprise ranges ofnode angle value in a motion profile.
 10. The computer-based system ofclaim 8, wherein the computer-based system does not comprise a motionprofile, the motion profile further comprising ranges of node anglevalue or displacement experienced for one or more exercises.
 11. Thecomputer-based system of claim 8, wherein the computer-based system doesnot comprise a database of previously recorded motion profiles.
 12. Thecomputer-based system of claim 8, wherein the computer-based system doesnot determine a rehab/treatment schedule.
 13. The computer-based systemof claim 8, wherein the processor being further configured to performthe skeletal tracking and calculating of the joint angles at thedifferent time points and the processor then being further configured toanalyze a gait of the subject, an angle of the knee joint of thesubject, jumping and the landing mechanics of the subject in furtherreference to other variables including biological gender of the subject,sport played by the subject, and/or previous injury of the subject. 14.The computer-based system of claim 8, wherein the processor beingfurther configured to determine the prediction of future injury to theknee anterior cruciate ligament of the subject from images captured by ahand-held imaging system.